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ABSTBACI 

This report presents a case $tudy of prograas that 
prdvide either foraal or in£o(|al' training, j|or eccnonically ' 
^^sadvantaged youth firpB fourteen to tventy*cne years of age at six 
(Aunity action agencies in the strate of Ohic. The folloving 

LBS involvi-ng the six agencies vere exaained: inschool public 
)r prograas funded by the Coaprehensive Eiployient Training Act - 
1^) ; out-of*school publix: and private sector ^prograas funded by 
out-of*school public and private' sector prograas funded by the 
lity Services Adainistration; and the Departaent of Labor's Job 
piograa. Data collected through tvo sets of on-^site interviews 
i^gency' adainistratcrs, job couns^^lors^^ job supervisc^rs, and 
par'ticipants^is ^uaaarized and presented in tablf and 
Lve fora in teras of strengths and weaknesses inherent in the. 
^genc^S and ^prograas. ifbcus isi on four facets of agency and p^ograa 
\«n: context, ideirtif icatiCD,^ intervention, and evaluation, 
idations grouped according to t^e \^our facets are present^ed 
tt^de iaproving ccaaunication b^tveen agencies ^and 
J^iening ti^e counseling functigns^ in agency p^rograaS. Apperndirxes 
'itt€lud#|4escription of aetbodological factors that vere considered 
pi^^or #0^ iapleaenting the study, and coaplete Bfets of interviev 
scWduitijI. A coapanion voluae of innovative prograa Ideas culled froa 
rthp. stu4f is available separately. (BL) . " 
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FOREWORD 



We are pleased* to present this paper ^to' the Ohio Department^ of . 
Econortiit and Communitj^ Development (DECD). It is a result of a 
cooperative working relationship between The Center for Vocational 
Education' (CVE) and the DECD. It presents a carefully prepared 
assessment'of the '%Jtjate of the art" with regard tovyouth tv*aining 
programs adfninistered by six representative community -^ctipn agerjcies 
'in the State. of *0h{o. A compa>iion report, to this one— "A 
Sampler of Innovative Program Ideal for Ohio's Community Action' 
AgenciesVrhas alsi^been .prepare'd and fs available ffom the Ohio 
Department of Ecortomic and Coiwnunity Development:* 

The Center is indebted to Dr. Ray Lawton of the DECD for his 
support afrtdienthusiasm for this endeavor; Appreciation is extended ^ 
to Dr. Jame^ W^Altschuld (CVE) and Ms. Terese lerry (DECD) who . 
prepared. tyfs paper and to Ms/ Janice Lave who reviewed the draft. 
Appreciation .f5 also extended to the six cdinmuriity action ^agencies 
and thl^T staffs and program enrollees for participating in this study. 

• ■ ' r ' 

Jlinal.ly, a special note of thanks is extended to Ms. Manene 
Linton who typed the manuscript and Dr. Jerry Walker, Associate 
Director for Evaluation, whose division was responsible f on conducting 
tpis activity. 
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* I. INTRODUCTION ' ' * • 

' . . * ^ • ' \ 

*" J * » 

Ovemew of the Study 

- In the jptdte of Ohio, nuineirous local -Tand county coramu(rity % 
action age/ci^'s function to f5rovide important needed ser^cjp 4o » 
disadvantaged populations' in both urban and ruraV lacations. Although 
*many of these agencies operate through churches and other:instilut-^' 



tionsj^the maipr comprehensive programs are found in 48 agencies 
funded by the uommunf ty Services Administration of the Federal 
government wi^ technical^ assisijince Trom the. Ohio Dep^^t^;tment of 
Economic. and Conmunity Development— Office of Human Services, 
.These agencies offer programs in 25 areas such as healthy day care, 
legal , services, educ^ition, youtl^s.ervices opportunitfes , etc. 

The majority- of funds for^ these programs come from federal sources 

• . \ ' . „ ^ 

with some additional support from state sources. 

^1 " ' . * ^ * ' * • — 

^ Jhe focus of , this repprf is oh a case study of programs that 

provide TBither formal or informal training for youth^ from 14-21 

yiars^of age at .six (6) seledted agencies. Generally, these programs 

e^ompass facets of e[ducation7wo/k^ experience, on the job training* 

coifhselirtg, vocational- tra5ning and the lipgfddtng of basic * , 

skills. The^selected agency sites were 6ach visited twite during 

the project witti on-site interviews occurring each tjme^. Addi- 

.tioniTly, existing local, clocuirientation was examined and analyzed 

to thfr-extent that time permitted ' . - \ 



The '.intent of this case study was to describe ;tKe state-of the 
art ifi youth" oriented tr'aininguro^grams based upon inputs from six 
agencies.. Strengths arjd weaknesses in existing programs^ were ^ 
identified and used as the basis for deteYmining Improvempnts that ' ' 
cpuld be effected -^n the future. - .Suppoirt service needs of.i^the agencies 
were also identified*! i Because this is a case study, the conclusion| 
should be viewed a^ preliminary and tentative in natiire. The 
conclusions should be probably thought^of as working suggestions or 
ideas^rather than being hard and fast findings. 

A Speci.al Note About Organization of the Report ^ * 

^ As "^'ndi^ated in the Table of Contents, -the report i-s .organized 
into six major sections and Appendices, /^endix 1 is designed to 
acquaint the reader wtth methodological .constderations and^i-ssues . 
pertaining to thp specific methodology '^mpl eyed in this study. 
Th.e reader is'encoura^d .to peruse it a^ 
of the report. ' 



ter completing ;the major text 



5R0UND- FACTOR? ^ 



■ ■ ' ■ -r 

. , ■ Jl. BACfffiROUN 

WKy Was the Study Initiated? \ . \^ ^ • 

An initial" question that could'be raised in conducting tf^s'.study • 

is, '^What factors or forces have led to the need for^e study and 

actuijjly helped .to make it a'realUy?" The best answer to .this question 

is that the study is the result of a combination of factors. First, 

the amount of funds expended for youth training programs that come, 

♦ 

under the jurisdiction of community action agencies i^^extensi ye. 
Mai^y community action agencies in Ohio administer Cdmprehensi ve 
Employment ^Training Act fiJnds:' Under several titles (I^and III) of 
the Act, community action agencies operate programs which are targeted - 
primarily for youth in the general 'a,g6 range of 14-21. In the past 
year, as examples, approximately three quarters of a mill ion. dol Tars 
of CETA funds were administered by the Erie Hurdn Community Action 
Commission and-approximately $2,000,000 by the Communjty Action 
Organization of Scioto ^County. While not all community action agencies 

administer CETA progr^ims, many do and the above examples if multiplied 

», » ' * * 
across many agencies serve to illustrate. the magnitude of funds 

. . .* ^ ^ • . ' ■ 

allocated -in this area. Additional, work experience programs funded ' 

by' otf|er sources are^also pperated b^ the agencies/ 

The exp^diture of large amounts of public monies is almost always 

now Ipsoifipanied B^^^ ^:he broadest sense of 

the term. That is, the accountability of programs must incl.ude 

measures and indications of success/impact as well a^ ;statements 6f 

how monies and resources were spent and us'ed. This has been reflected 

in national legjslatipn regarding education (ESEA, 1965) and in\ 

various allowable budget* categorizes for community action a^enc^es ^ 

as specified by thetiffice of Economic Opportunity. One^might ^a^ . 

that. "program accountability is a sigh of oyr times." t 




Secondly, this;Study js partly attributable to a genuine desir6 
to Improve existing programs; to develop new/ innovative program^; . 
and to improve- the overall operation of conmunity action-^ageficfes. ^ * 
The above concerns are representa^tive of^the .per'ceptions of the Ohio 
'Department of Economic and Communitx Development as wejl as a sizeable ' 
number of th^ agencies 'that participated in this study. ' * ' / 

. Thirdly, the. study reflects an understanding on the part of 'the 
Oh-io Department and the' participating agencies that there are variouS\A 

( • • . 

alternatives for studying existing programs:* - For eJfemple, the Ohio 

Department or ^' committee 'from the agencies could' h^ve, examined 

existing programs and program operations.' While thfs strategy is 

g ^ * 
qufte possible,, it do^es. have a noticeable deficiency.' Individuals 

comprising. the study group may be "too clo^e" to the problems and 

issi/Bs and be less objective than .those wh# are -di verged or separated . 

frofiithe agencies. The need to conduct the study by an . External 

party thus becomes apparent. • By contract this external party was * ~ 

the Evaluation Division of The -Center for Vocational Education at ^ 

The Ohio' State University and specifically one of its resxjient evaluatQ»^ 

who was designated t;o carry out the study; fo facilitate the" study, 

the Department of Economic and Community. Development (DECD) assigned 

one, of its field representatives to help coordinate activities* and . / ^ 

Jio work closely with The fenter for Vocational Education. Both'pf/the' 

above individuals shared in the authorship of this report. ^ 



Initial Planning Efforts ' ^ , • ^ . 

^ The actual decision to implement the study *Was the re*ult of 

several jneetings held in the spring and^ummer of 1976. These *" 

meetings were as follows: . ^ . # . * / 

Initial discussion meetings in'volvi/ig DECD •staff and staff 
from CVE ' ■ . N ' . • 

Contacts, meetings, discussions- between OECD staff and*' 
potential participant age'ncies; and • ' ' 

7 A major planning meetinjgrin July that included DECD staff, 
CVE staff and participating. agencies. y . 

Tbe last fneeting was especially inportant inasmuch cis It- provided « 
an opportunity for all involved parties tQ) discuss the. nature of the 
study. Potential ageas of misunderstShding were clarified and resolved^ 
The interactions' among ^all parties were explained and tjie cooperation^ 
necessary for carrying out the stu^ly was. explained. 
^ J The Six agencies that were* invited to and did actually participate 
in the study ^ere selected on the basis of .numerous criteria.. The 
criteria, we rq: type of area. being served by the agency (i.e., 
metropolitan* urban, and r.ural with two agencies per eacFi typ6 of area); 
innovativeness of program as initially judged by the professional 
staff .of the Qhio Department of Economic and Community Development; 
and agency willingness to participate. Table 1 contains a listing of 
the agencies that participated and the ^pl of area they^jjepresent. 
' . It should be notec^ that the jnetropol i tan , urban* and. rural 
dist>nctions sb^wn in Table 1 are 'not absolute ones. 'Rather, they 
represent some fine gray lines between the agencies bas«i upon 



professional ju^gmemt. Hopefully!, the agencies selected are 



Table 1 



Community Action Agencies .that Participated 
nn the Study by Type of Area Represented^^ 



T^pe of Area 
Metropolitan 



Agfericy Title . ^ - . £ity 

^Coun.cil for Economic ^ • Cleveland 

Opportunities fn! Greater . - 
Cleveland (CEQGQ) ' , . ^ / ' 

Columbus Metropolitan Area ' Columbus 

Community Action Organizatioa , ^. * 



Urban 



Erie-Huron Counties Community 
Action Comnission 

Communal ty Afiti on Orgapizatign 
of Scioto County, Inc. 



^ Sandusky 
Portsmo^uth 



Rural 



Adams-Brown Qommuntty Decatur 
Action Program 

/Hancock, Hardin, WyandQt and'. -/^ FindlaV 
' Putnam Community ActioVi ' " * • 

Comnission ^ ^ ' 



represerltative of a cross section, of^ coirtmunity action agencies, based 

' ' ■-. < \ -a:v>; ■ . ■ . 

upon typ6 af afeavserved. .At- the end. oT; SeptfenTber\, the contract , \ 

between the Ohio Department of! EtorTbralc'^and^ljCdtnrmjnltyv'Development iand 

The Certter. for Vocational ;Cducatior( (^p*e,^0h^^ 

(. • ' * '^^ft^^ ' ^ 

was formairy signed and thfe project was' started ^t..the beginning 

^ • • • ' ^ 

of October, / ^ ; 



Objectives of the Study ' ^ 

L Listed below are the objectives of this stWy: • ' , 

• 1 . To describe youth orieated- training programs operated by * 
'community action agencies In the state .of Ohio. The/ 
descKiption will ijrclude the dantext, i n wh1^ the agenctes/^ 
• programs operate; the tnanrter in which* paj:€icipants& are 
identified; the nature of^.the specific pr^gt'irns (or interven- 
tiojris);.^nd program evaluation. > ^ ' v ' 



2. To identify unique, innovative programs, . i .e.- , ^promising ^ -^^ 
^ programs that hdperulT^ could be genel^alized tb other ^settings. 

3. To. obtain recommendations for "j^iproA^ment from thfe six (6) 
participating sites. * \ ^ • ■ , . 

4. 7o generate, based upon inform^tion^coll^cted^ for objectives 
one through thrfee, two r^ol^ts.- ~ - ' ; . • 

' ' • • • ' i ' ' ' 

\ - , The first report*(of whjch this writinq-js a part)J.s a 
summar>^of the stud^nd its resyilts.-^ p ^ 

- 'The se^onV report ,wi 11 highlight ir|nDvati\« programs that - 
could be generalized acnoss agencies. ' It will basically- 
be in the format of a brief "farnpler" of iiinovative . ,^ 
programs and suggestions* for improving agency apd program ^ 
ope rati on Y The agencies will have the opportunity .tq choose/ 
s'elect any of the ideas ^in this -report for their specific 
sit^ati]ori if they see fft to do:*so; ^ * ^ 

Jhe procedures for conducting tfte study are described in the next . ' 

section of this report. ^. / * . * \ 



..III. PROCEDURES FOR-CONDUCTING' -THE STUDY . , 

*.'■•• ' - " ' ' - . ' . . -■ 
Two'nWave Interview Approach • • > 

-To ^tudy the youth oriente.d tra1n^»^rograms in-the coimiuni^ • 
■ • • , ' _ •' ' ' . ' . 

action agencifes, each agency was visited twice. During the initial 

visit; only key adrairfiS^trative personnel were interviewed.* 'Usually 

tftese interview's included the executive director of* the agency aifl the 

•- ■ ' ■ • . . o ^ - ■ ■ - Z'^. ■ ■. 

manpower director. Tfie interview was intensive in nature and required 

between.2-3 hours t© complete. • , • " ' 

Based upon the'findings from the init,ial intervifewS conducted 

at the'si)o sites the scope of interviewing during fhe secfonjd. visit.' 

was broadened ta include in sepSarate 'interviews;, counselors;' work 

experiefice -supervisors; and progf am parti ci|iants^ The latter two 
^^-'*^ ' ^ ^ ^ ' 

sets of individuals wei^'6 interviewed (directly on ;the job site* .All 

of the interviews exofept.for the (jsA d^cribed b^elow were' brief anA » 

generally required one hour "Qr less of time. Invad^jtloti, a secdnd - 

.short interview was again conducted wi th the key agency Sidiuinistrative 
per'spnneT^ This interview genera.Uy required one hour to ."complete.- - 
The plan for interviewing and the time requfWId are summarized* . 

,:in Table 2. One fact that <:an .be inferred from Table 2 is that , 
access the sites a miniraum tott'l o-f 42 hours (6 >ites 'tirhes 7 hours)' 
of interviews wars- planned for.' The initial interview was conducted 
jointly by the two au^Jjors. or this , report. 'All subsequent interviews 
we^ cohducted byJple authbrsbHbwt* on .an tndividua] basis. This was 

*se6n as the most feasibli' approach to accompiJishing the interviewing 
within a reasonable period of time in a "work day. ' ' •. 



• ' . , . ' ' Table 2 / ^ 

Thfe. Jntervlew Plan for Iriitial and . 
* ^ Second Site Visits to -a Single Site 



• ' ' . ^ Total 

^minis'tratoi^s Cpynselors Supervisors " Participants -Time' 



Ifltti^V** Exemittve- , *- 
Visit Director and/ . , 
.... • -dr Agency ". . ' 

Manpower. 
- , Director - 
(2,-3 lirs.). ' 



2-3 
hours 



Se<!;Dnd " ' Executive 
Visit Director and/ 
or Agency - 
. Manpower 
) Directpr a- 
\' (1 hrV) ^ 



2 (1/2 hrv 2 (1/2 hv. 4 (1/2 hr. 

per " ' ^ or less or less- 

counselor) per per 

supervisor) participant) 



-4 . 
hours 



\ 



14 



■ --10- 

> • • ' . - . 

Where possible, the plan cplled for interVsiewing at least two , 
counsel ors^'frojn di fferent programs , two- supervisors' from-the different 
programs and two participants per program.. In several instances, 
the agency was^jply operating one main youth oriented training program 
an'dhe^ce'the plan was -mod^^^^cordingly. ..The actual types an^ " 
ntj wbet i'i of ;i ridi v' i4ua.U-i^ervi e.wfid, are f^gndMri the results section. 

• ■ ' . ' ' 7\ 

Interview, Schedules/Forms ^ 

Th^ interview forms are appended to this report in Appendices 
Il-yi. The form for the initial site visit (Appendix II) was . 

developed according, to the following concejJts: J ; - 

'■ , . ' '■ 

- Context refers to'th,ose variables and factors which, define 
_ tfje milieu in which 'the community action'agency exij5ts. 
These would include: type of population served; general types 
of programs .operated l)y the agency; economicfactorsratfecting 

- ' the cDitmuriity served; etc. 



- ' identification re/ers to the procedures and techniques by / - 
which potential pairtici pants are defined, screened, etc and^ 
"eventually admitted to a youth training program. , 

Intervention refers to the techn>ques. and methods by.wh.ich 
pjifgfams are operated and managed. Programs can be characterized 
.by their (*|^tion, purpose, actual activities and so fgrth. • 

- Evaluation refers^ to the processes and methods by which' a 
program staff obtains and utilizes information regardina^he ^ ^ 
effectiveness of programs. • f • 

The initial interview contained 15 questions across the above four 

areas .'N^ addition, three general overall qjjestfons relattn^g to ^: 

pregram strengths,' program "weaknesses and suggestions for furthgj?^ 

program study .were included in the i/itefview. The initijtl interview . 

was, only utilized with key agencv adm-fnistrators. -A 



' T^e second interview (Appendix III) wT*h key administrative * ^ 
staff was somewhat shorter in length and focused on cohcernsr qi|tte^ 
different from the first .ojie, the eighff .cjugstions of';!this interview 
primarily:tlealtllrth issues, or concerns identified ^uring thp, firsts . 
^ ^tnterview as being cdmmon acrojs the six\sites.' One 'quest ion:, however 
J^s_^1J:a_spec+f?tr'and rejated to innovative types of *programs operated 

0 • 

The interviews with counselors, supervisors and participants were 
very brief in^length and focused on the .fol1|^1ng types^ of variables: 
problems^ encountered with the program; program strengths; satisfaction 

"iflfith program activities; suggested program modification; aKd so forth. 
(See Appendices IV-VI). All intefrviews were conducted with the stated 

4 assurance? . of CVE and DECD that all information wis^confidential and 
would only 'be used in "grouped data analyses." 

Other Data Sources ' . - 



In addition to ttie interviews printed sources of data were 

^ collected at each sfte. These sources .includSd reports^ pamph;]ets, 

evaluation fqrms and other similar rna^eVials.\4jyTo the extent possible, 

these sources were reviewed and the faefs they provided, ha^ be€in 

incorporated into the .study. ^ ' / * ^ 

• . ' • - . : 

Data Analysis ^ ^ * \ . • * . ' - 

The basic mode of analysis was the callatlon of interview data 

and other -data 'across the'six sites.^ Where appropriate, frequencjf 

• counts and tables were prepared 'which summarize the findings of .t-he ^. 
study. Interpretations of the data and conclusions are inqluded 

•with the data summary. (For more detail ami backgrounds factors re- 
gardfng the i'nterpretation of case study results, see Appendix I). / 



. : I^. RESULTS ' s / . 

• . • / 

.Sampling ^Summary for the Initial and Second V^istts to the Six Sites 

in Table 3, the actual sample used in s'tudying the youth oriented 
training programs of agencies included in this -study is presented. 
•Thfe table contains the sample from botlti the first and second site 
visits. * ^ , . ^ 

Simple addition indicates 4:hat 62 separate interviews were 
conducted by the authors of this report. It would have been desirable 
to have interviewed more ^counselors. and participants, but due to the * 
press, of time, distance, and local ashetiulfefs this was not possible. 

The table further indic^ites that .in m^y -instances only out- , 
.of-school prograins were, observed. Although this is partly an incidental 
resuH of scheduling di/ficulties it'does reHfet an emerging problem' . 
for -communit^ a^ion agencies since 'some of the agifincies have been- 
fprced^tof' curtail ^their youth training programs due to- loss of funds. 
•4This*particular point will be discussed in greater detail in the' next 
this report..). Lastly, although not specifically described ' 



se/^ion of 



in the', table, it should be noted that the on-^site visits "took place* , 
from early October up to the beginning of February. 



Context' - Results' and Discussion , , ^ ' 

In Table 4, 4 summary of the agency contextual factors is presented. 
This summary wa*s^geDera1;ed from two sources— first and second inter- 
views with key agency administrators and It brief review of materials 
(that th^privide4 to the interviewers. Analysis of the table, tends 
,to-reveaTi||pme^J_pferesting facts about the nature- of the agencies. , 
These facts ifre" listed below. . ' - ' ^ 



Table 3' 



Actual Sample Interviewed During the Irjitial: 
affd S%ond Visits to the Sites 



T" ] ' [ ^ 7 ' \ ^ Training/ 

^ • • \Work 

Strata/Agency^ Initial ViMt Sample- Second Vfsife Sample Experience 

. . ' . , . ^ ' Programs 



♦ Hetrppolitan 
\ Agency 1 , 



/\gency- 2 



Program Director r • Program Cbor'dinator Out-of- 

Program^rdinato^. .1 'counselor school 

. 2 supervisors prp^jram-. 
» '2 parti ei|jS?fts 



Deputy Executive 
Director^ 



Deputy Exec* Dir. Out-of- 
Program Director • school 
Couriselor Superv. ^ program 
' '1 counselor ^ / 
3 supervisors 
' , 3 participants, ' ' . 



Ur5an ^"^^ ' 

Agency 1 , Projgram Director, 



Agfency 2 



Program Direfctor . Out-of- 
'*2- counselors ' school 
} s{lpervisor * pro'gram 
1 participant . 



Executive Director 
Pnogram Director * 



•Executive .Director 
^Program Director 

2 coOnselors 

3 supervisors * " 
4' partjcipanta 



Fti-s 
Out-of 
school 



iRura> r 



1 



Agency 1 



ExecutivelSi rector* 
Program 'Di rector 



• -Aigericy 2 



tiJtalS* 



• Executive ^Director 
J Program 



Executive Director 
3 counselors 
Z supervisors -\ 
5 payii ci pants. 




Ex^cuti veJJi rector 
Program Director 
ounselors \ 
. 3 luj^enMsors 
^2 participants 



^1 

' 4. Executive Dir.**s 
5 Program Dlrdcto^rs 
prd'pram Coord . Cs 



In-school 
school . . 



In-school 

Out-of- 

school 





4 Executive Dir. *a 
4 Program Di'rec^tors 
1 Program Coordinator 
1 Counselor Supervisor 

*n Counselorjs 

14 Supervisors ; 

•17 Participants 



Table 4 



\ Agency Context Surrmary as derived from First and Second Interviews with Agency^ Administrators 



Type/Agency XSA Funds* > 



^ Total ^ . ' CETA ' 
Agfency Funds Programs**^ 



Population/Type 



Other eomnjents/ 
Descriptors . 



Me'tropoTitain 
Agency 1 



flgency 2' 



$UO00,0O0*** •'$4,3Z6.882 



$1.,969',000 



• 116,276,741 



/ 



-7^ 



Out of School » Large metropolitan 
(QfTA^ Intake) ^ arear with the 
^; ' / \target population • 
^ \ > being primarily^^, 
inner city blacks* 
'and'appalachiart, • 
' *' ' ' white". . - ' • 



- Multi -faceted i 
agency which ^ 

^ iiandles funds 

• from HEW, CSA, 

* etc. 

r Several not for 
profit companies 
Cca'n.wash,.pi?int 
shop) 

- 7 'neighborhood ' 
. service centers 

(OutreachT. 



I. 



Larg^' metropolitan , - 6 neighborhood' 
• area wi*. ^he jtarge^^^. j^nters . 
. poputatTpn^ beihg ' \ I --. Agency is'> p^ss 
>inner city .-blacks,', ^-through or co- 



whites, arid 
-Puerto Ricanfe.'^ 



ordinatifig 
mechan^ism for 
various- groups,^ 
e.gv , Seniors* of 
Ohio, Inc.; legal 
services society," 
• etc. 



I 
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* CSA = Community Servij:es-/!ydininistratlon.. \ ^; ' ' ; "• ' ' , \ - ^ * ' . ' ^ , 

** CETA = Comp»:eli€hstve Employment Training Act ; ' - ^'"^^ • . '• ' 

*** Many figures are approximate and should only faq^ viewed^ a giiicle for ^understanding tRe_pdhtext oT^the., ^^-y 
agencies. * , • , . • J 
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Tablie 4, continued 



Type/Agency \ CSA 'Fund^ 



' Totil CETA 
Agency Funds Programs 



t Population/Type 



'Other Cofnments/ 
Descriptors ' 



Urban ^ % 
Agency. 1 



Agency 2 



$152,000 . 



40,000 ; 



$l,l'50^000 



$3,000,000 . 



In-school 

Out-of-school 

SPEDY 



In-school 
Out-of-school 
SPEDY \ ^, 



.gO?odo in one , - 

county which is v 
tnosFly urWn- 
suburb€m-(42-WS&^--^ 
.bUcMU.5D^QQ0,ilU^^,; 
the other which has as 
a target population ' 
poor rural whites, 
migrant fjimiTfes 
are 1-1 1/Z« of 
the population*, 

80,000 with the 
biggest city being 
about 25,000,000'/^ 
Mostly the area is 
rurffl , ap£alachian - 
and poor.. About \' " 
25^30% is con- " ' . - 
side red at the- 
poverty leveL 



3 neighborhood . 
centef s are ' 
operated* iruthe 
2-ec*iffrttes^ ^ 
^lultl faceted 
programs . 



One of the higher 
incidences of 
poverty 'in the 
state of ^hib 
Multi faceted. ^ 
programs 

Trartst)6i^tatidn / • 
problem? % 



Table 4, 'continued 



Type/Agency 



CSA Funds* 



Total 
AgenfcyfFunds 



CETA 
Program^** 



Population/Type 



Other Comments/ 
Descriptors 



Rural 



^ Agency 1 



$150,000 . 



$1,300,000 



In school 
Out bf school 
SPEDY 



Agency 2 



$96 ,000 



$1 ,800,000' 



f 



In school 
Out 6f .seftool 
SPEDY f 



49,000 tota^y 
ru'raK largest 
city is 2,900. 
The area is ap- 
pa/lachian and poor\ 
15% or more - ADC. 



150,000 rural 
except in one 
coiinty. Poor 
popuratiofcis' ^ 
primaril^appala- 
Chi an whites 
(Wrtll + after) , 
and migrant/, 
settled^'out > 
migrants 



Multi faceted program^ 
Outreath aides 
in local com- / 
muni ties/areas . 
High incidence 
of h'igh school 
dropouts 
TrarisporlNition 
problems^ 

agency operates 
multiv- ^ 
a^eted programs«r 
'^herfe ^re sea^orfal 
problems due to 
mi grant 'families 
arriving in the 
area from 
April to October 
(also discrimina- 
tion towar^d . * ^ ~ 
migrant is , 
.apparent) 
Bilingual staff 
are needed • • 
Many appalachian 
whites - 2nd- 
3rd generation 
welfare farpilies 
Transportation 
problems 
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^. Complexity of Operation . All tlje agencies run , numerous 

programs Qn diverse^ ar^eas such as alcoholism, legal services, 
youth train^'ng, elderly assistance, .and saon. The agencies" 

I are, in effect,' parti.al coordinators of local sociaKservices 
to conmunities. ^ ' ^ ' 



2. , Level of Funding .- The complexity .of operation is underscored 

when the level of •funding^ is e;Camined. Even the smallest 
, Agency administers well over a /million dollars in funding with 
•' the largest operating budget being greater than 16 million : 
dollars. Since^fchese fun^s come from diverse soUrc^s, th6^ 
' ' - -agencies^aire ha^itig to be fafnly ^cti.ve in seelcing funds. ^ 

3. Population Served .^ The agenoles do indeed vary in terms of th 
population^ serfK. Five different groups were identified^ 

in this study:; rural poor;, inner city blacks, appalachian 
whites; Puerto Ricans; aniTraigrant fai^ workers.* The unique 
features of these populations make it difficult; to define 
programs tlratirill work eqi/ally well in all settings. 

CETA Funding . 'The tat^e niay indicate .that the pattern of 
funding is changing across the agencies. ^ Thre 
metropolitan agencies hayeno^ver the/ past several years seen 
their control/involvemeryc in Comprehensive Employment Training 
Act (CETA) programs decrease ^nd/djr cease to exist at ell. 
The locus of control of CETA has undergone a shift from com- 
munity action to city government jn the metropolitan areas. 
Undoubtedly, there is a number of reasons for this change 
including poli tical .ones. ' These/ factors are discussed in 
greater detail later in thts report. " - ^ 

The strengths and problems wUhin xhe.x:jontext that the agenctes • 

operate in all summarized in Table 5./ -This information was derived* 

partly,, from an analysis of the conte/lctual information as shown jh 

Table 4 and from other informatioff collected during interviews with , 

the agency administrators. As shown in the table community relations 

is judged to be a,^^trong point of' age/icy operation/ All the 

'agencies are truly part^f the community in which they exist. They 

are all in effect decenti4lized and operate community or outreetch- 

centers. While the information supporting thisf strength was derived 

from interviews and not directly observed, it does appear to be 

a strength of the cigencies.'^ 
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SummaryAof Main ConftextuaV 'Strengths and Problems Observed- 



Strengths 



Problepis 



Contfeunity relations 
Coord i ija t i on o^ programs 



t- ^^^^ . 



- Inrtfer-agency linkagjaa/coorcl^n^ition 
Transportation problems (especia:lly 
in rural areas i 

- CETA funding Toss . 4 



Coordination of programs* is also judgerf; to'be a strength by , 
virirCe of the* fact that all agencies are striving to interrelate 
programs/ Vouthful workers, for example, may 'run sunnier playground/ 
recreation programs or they may be on work teams assisting the elder*ly. 
Winterization programs,^ i .e. ,^improving^ th^ insi^atit)n of low income 
dwellings, are frequently accomplished by using youthful workers. To 
the extent |R)ssible, 1;he coimiunity action agencies also* employ workers* 
directlj^ on their p^remises. 

^ ,0n the iiegSTTve side a^ problem occurs in the area of inter agency 
(social welfare, community action, municipal, government) cooperation , 
and coordination. The coordination of services across agencies 
within, a number of communities is not particularly good; Where this 
problem is taking place several^ community action agencies have now 
joined in membership with other cominunity agencies In coordinating- 
typ^s af area coiwMttees. These gfoups re trying to if^educe excessive, 
unnecessary agency overlap, increase efficiency and provide a better 
set of services tp low income and disa<lvantaged groups. ' ^ 



* One other strength in terms of context was observed in two agencies 
which are part of COAD--afi appalachian regidn cooperative.^ This 
' cooperative seeks funding, writes joint proposals, sh^wees problems 
and ideas, etc. This t^p'e of cooperative based upon rtiutual needs/ 
and problems may have value for other are^s^of the state. 
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;W^Over time, these committees coirld produce tretnendous ben6fit§ for 

• . disadvJtn^aged^n They could aid in preventing such things - 

^as: confusion over who to go to for heip; back and forth and forth * 

and b'&ck referrals with no r^esuUs and ' probable clisenchahtment with 

'^'the system; unclear agendy functibns^anjl boundaries;! similar prqgram 

names; the welfare/ community action maze, lack of communication between 

' ^ - • » ^ ' \ 
agencies; and so forth.. A beginning, has been made^l^ this aresli— mor^, 

needs to b'eMone. 

^ Harsh tr^nsport^ation problems were observed in all the rural 

community action agencies that participated in thsls study. Associated. 

' with trahpspOftation are factprs such as physical isolation, lacic of 

roleNrodel? and lack of job/work ^site opporturtit^es. Sdme token 

suggestiqns fpr improvement in this reg^nrd will be .offered in subsequent 

.-.sections of fhe report but the difficulty is considered to be pe'rvasive 

• - _ 

and not. easily overcome. • ' • . v 

The last, problem, CETA fypding loss* is»jperhaps ^e most > 
- critical one. In metropolitan area§ the comnunity action agencies 

are, for the most part, no longer*5rece1vtr*g CETA funds. In rural • 

• •» ' , * ' ' 

areas, this has not occurred but it 'may take place' In the '.future. 

This shift is control of CETA is noticeable and as a result,^ \ 

t . t - ■. ,_ - - , • 

•the CET/Hprogramjnay be^more polUicized than it formally was. 

-^^^^^ArJ^mportant aspect of thlT^'ft is that the agencies now in " 

charge, of CETA Tunds hai^e a pr^mar^ function that supercedes the'. ^ . 

"trainifig-fotfction. That i!^heir\godl is to run tl\e city and 

■ provide its citizens with services necessary for their sur^al , 

' . ' ■ %♦ ■ 

safety, etc. CETA liiay provide funds for workers who do asswt in 

*the p-rovis-ion of suchservices. Thls^^.well ind good: but it 

.the prim^rylpurpose of CETA funding... CEJA funding is primar.il^esigned 

- : . - , ..27V - ; • ./ 



for^^e puf'pose jjf Training with service as a logical extension of the 
purpose but^a secondary one at best. If the use oi CETA funds 
basjcally provides service rather than a meaningful training experience 

for an individual it is not meeting the goal of the programs. In 

* . » - 

addition, CETA funds are not intended to either^upplement or supplant 
locally funded governlnent services. ' : ^ • ^ - . ' 

(Reasons other than political ones could be postulated for the 
shift in control o'^^;ftjnd§. These might include: %poor fiscal manage- 
ment by the comnunity action agenficies; lack of innovative training 
opportunities; inefficient programs; etc. It is however, beyond the 
scope of this study to examine such potential catfsal factors.) 

Identificat ion - Results and Discussion * ' - - 

— » ' — ^ ^ , 

Identification really refers -tb two separate types of activities:. ' 
informing ("getting the word out") potential' participants of x the 
various, opportunities available to'thfem; and the actual- process of 
sel'ecti|U[ individuals for programs. Across" the six agencies a common 
set: or both informational and i^d^ntification, procedures was observed. 
These in conjunction with strengths antf problems are summarized 
in Table 6* . ' ^ , ^ 

The procedures de^ribed in the tabJe^seem to work fairly well 
inasmuch ^s most agencies report a surj)lus o^ applicants for available 
programs. According to Igency reports, almost all of the applicants 
are ^i|RT below the. poverty guidelines and do generally come front 
. targeteq^j'sadvantaged groups. The agencies do report turning away 
applicant? who, al though'. needy , falV into'^a classification slightly 
abojfe Jl:he 'poverty guidelines. . *\ * /4 * 



Table 6 



Summary of Participant Identific-atlon Procedures,* 
Strengths and Problems as Derived from Interviews 
_^ with Agency Administrators* §^ 



Procedures 



Strengths 



• • tPro 




Informational 

Word of mouth 

- Posters in schools 

- Radio/TV 

- Flyers 

c' Referrals 

- SchooT contacts 
* - Newspapers * * 

- Outreach 



Selectjion 



First come/ 
first served 
Those with 
greatest need 
Interviews/ 
Mom visirts 
Department of ' 
labor criteria 



All agencies^ seem 
t<f employ multi- 
faceted procedures 
Many 1nter/intra 
agency referrals ' 



Genei^ally all- 
-agencies adhere to 
DOL ^target* popula- 
tion 'guidelines 



More' applicants 
identified than^ 
can be sejrviced 



r 



Most agencies use a combination of procedures afid the order of 
presentation of procedures ^n the table does no^t reflect an 
emphasis 6n any one. procedure*. -All are used pommonly. - 



One set -of issues/consi(^erations not described* in- the table 
relates ttrthe procedures for idertt-lfylng/selecting program wgrk si'tes. 
A problem encountered at severaT^a^encjes was a concern over the ^ 
degree to wMch specific work^, sites really provi4ed parti cit)a'nts with 
meaningful training and^the degree to which participant problems were 
underHood, ' Partly, this is ^ "problem of continyous orientation and 
dialogue between the agency and* the w^rk^site. Partly, however, it is 
a problem of initial site selectioji. At present, this is usually^ 
accompli shedr-by job coordinators, proc|ram .directors or supervisors 
contacting sites or learning abo^t sites via their work in the field. 
The professional judgments of these^fndi vidua Is have generally beSn 

In the same instance,. It is possible that excellent traijiing 
sites'are averlooked. This coyld be a* reflection of publicity 
caf^igns dirfected toward prospective participants rather than * 
prospective^ employers both In: the -public iand private) - sector.* 

Interjfentionl- Results and Discussion . ' ^ ' ^ 

In studying the actual trafning/work experiences operated by the ^ 
agencies, a variety of individuals and groups was interviewed. . As. 
described, this was dofte to obtain ^i, comprehensive view of youth 
oriented training pro|rafn#; The* results of these> interviews are found 



bfes r 



in the following 'tab! 



* One agency iis*currently pursuing this , course of action. 
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Table' 7: General , Program Summary 

Table 8; Counselor Perceptions of Programs ^ ' . 

Table 9: Supervisor Perceptions of Programs * > " 

Table 10: Participant Perceptions of Programs 
Table 11^- Innovative Programs/ Ideas- . 
-Table 12: Summary of Program Strengths and Problems 
' Following the tables is k short discussion of the results^" Table 11 * * ' 
wijll not be discussid in any. detail in th-is report inasmuch as it 
will t)e the substance of another product prepared for the Department 
of Economic and Conmunity Development. ^ . ^ 

First, in reviewing Tables 7^through 12, the reader should be 
.aware of the procedures ujsed in constructing them. The tables are 
a condensation, of information collected from partially open-ended 
interviews. To collate data for tabular display required that the 

ess'ential idea stated by the "interviewee b^ummaFized. Interviewee 

■- ' ' 

'comments are thus not-inasmuch- dg'tail as originSiny given. Interviewer v 

« ♦ - ■ ■ 

judgmefit was also used ^n selecting the most pertinent comments- for 

. listing in the tatles. Lastly since the .sample size per table was 

* # 1 ^ 

small, no attempt was .made to 'separate out the different types of- 
progr^ms. • -. * 

Table 7 indicates that six baSic types of programs are in opera- 
tion at the six sites. The six programs are: CETA in-school program; 
CETA out-of-scfTOO| program; CETA summer program (SPEDY); the Depart- 
ment of Labor Job'C«iH)s program; CSA out-of-school (private and public 
sector*) programs; and CEfA' put-of-school (private sector) 'programs. 
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Table 7 



General Y(|ith Training Program Sunm^ry as Deri,ved from AgenCy Intervi 

Instructional Strategy-. 



ews^ 



Funding Type of Average * . ; ' ^ 

Source * Progratn Durg t i on ^ Ob j ec ti ve ( s ) 



Evaluation 

Procedures 
r 



CETA. 



CETA 



In-sc^ool 



1 academic - 
year ^' 



To keep' students in 
school by providing 
positive part-time 
wor-k experience • 



r 



-Alinost all training 
is obtained hy work * 
supervisofs teach i-ng 
•students the job ^• 



Sup^rvisiQ^j eval uation 
(4 out of 5 agencies) 
Self ratings (1 agency) 
Counselor evaluations 
(4 agencies) 

ConferencV^^ agencies) 
•Various records 
(all agencies)* 



Out-bf- 
scliool . # 



Up^to 1 

calendar 

year 



To provide positive See^ab^ve 
work experiences 
foi^ out of school 
youths/young. adults 
To"deve1op joU • ' , 
survival and/<)r 
other skills 



See above 



I 



CETA 



1 



Summ^X> 
Program 



1 summer 



An extension- of 

' the'in-.schooh 
program isxcept 
that students c^n 
work more than a ' 
few hours part-time 

-"To\"keepL" stud^r^s 
off the streets 



above 



See above 
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Table 7, cDntlnJed 



Funding 
Soi^e 



Type of 
Program 



Average 
Duration 



ObrjectiveCs) 



Instructional Strategy Evaluation Procedures 



Department 
of tabor 



CSA . 
(Public 
r and 
private 
sectors) 



Job 

Corps** 



Dependent 
or\ type of 
training • ^ 
^(6 months - 
2 years) 



. To prov^ide skill, 
training in a 
residential camp 
setting ' 



Vocational school 
type of training 



Not^avallable (byt 
assumed to be skl'll 
providiency) 



Out-of- 
school 



Up to 1 

calendar 

year 



To provide posl- 
t'i ve work exper- • 
fences for out- 
of-school youths/ 
young adults 
To develop job * 
survival and/or . 
otJier skills 



Almost all training 
is obtairted by viork, ^ 
supervisors teaching 
students the jobs. 



- Supervisor ratings 
(1 agency) 

- Counselor ratings* 
(1 agehcy) 

- Conferences 
(1 agency^ 



I 



CETA Oyt-of- 

(Private school 
employment 
training) 



Up to 1 
-calendar 
year , 



To provfde skill 
training 



%>^ndent upon 
ifmiViHuail 
en^loyer 
contract 



^ As notejj' earlier^ not all programs ^re operant in each of the agencies that- parti^cipated in thts study. 
**~ Agencies do not directly handle the training aspects of' the program/ 
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As is noted in the table, no one agency operates all six programs. 
Four of the agencies operate CETA in school, out-of-school , and summer 

*i}ro^^rams as ma^er of course. In addition^ one of these four agencies 
operates the CETA private sector^^out-of-school program. One agencx^ 
operates only the CETA out-of-school program (and CETA intake ; 
functions) and is heavily involved in recruiting Ifc^ 'the Job Corps 
program? The last agency does not receive CETA funds tftid thys 

, utilizes CSA fundifig for its' programs. .The J()b Corps, CETA private 
sector, out-of-school , and the CSA programs receive minimal funds 
in comparison to the other CETA programs. * . . . • 

Within the corttext of these six programmatic areas, minor and 
major variations 6f program theme occur. Thejuajor variations , of theme, 
that is, those programs that are different a^d Innovative are • 
summarized in Table 11. Further detail regarding these variations , 
is given in the product entitled "A Sampler of Innovative Program 
Ideas fo»*..Ohio's Community Action Agencies" which accompanies this 

^reportv* » 

Other aspects of the programs- which are important are that: 
most program objectives are focused on fhe provision of positive ^ 
work experiences and the development of coping/ job survival skills; 
almost all training is dependent upon the specific job situation and 
supervisor; and a variety of evaluatiop techniques are used. 



* Altschuld, James W. and Terry. T. "A Sampler of Innovative Program 
Ideas for Ohio's Community 'Act ion Agencies^" The Center for 
Vocational Education, 1977. 
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The amount of training varies -greatly. In some situations, a 
supervisor wilV assist 'V participant in the learning of basic and 
adv*anced skills. Even with the* small sample i-nterviewed 'in this study 

j,HFew instances. of highly specialized train4hg were taking- place. 

*^^the other hand, some participants-receive only minimal training 
|g||p when/they are capable of learning' fairly complex technical 

ski-lls. This could be a. result ^f* numerous ^factors -such as: participant 

r 

.attitudes; supervisor attitudes^^and deficient^participant' educational 
Jjackgrounds. ^ ^. . ^ • . • 

•The format of the training while somewhat variable oi[ten follows 
a common pattern.^ In js^ qpe*to one fashion, the supervisor orients f 
the participant to' job duties, skills,* learning sjtuation, etc. and 
monitors the progress of the participants during their tenure on. the 
job. The-agency counselor visits the site iJeriqdically for discus- . 
sions with the supervisor and program participant. Interestingly, 
the participant is. -not an employee of th^ job site but rather is 
paid minimum wages ($2. 30. per hour) by the agency. Also, by regula- • 
tion CEtA employment funds (except In special circumstances) can 

only be allocated for use Sx\ governmental^ and. not for profit agencies. 

* I ' ' ' 

Hence,, there are limitations on the sites that. can actually be 

involved in the' trainijig and work experience programs. 



V 
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Jable 8 represents a summary of counselor perceptions of the youth 

training i^il^rams. One major 'finding from this tab|e js' that the 

counselors are either wSJl trained ^college degrees in sociology, 

education, and related fields) or/have considerable amoynt'of' 

^, practical exper^ience. Generally/ the counselors felt that th'Sir case 

' ^ / ' / - * ^ * 

loads were not too heavy and theV reported using /a variety of ' 

* ■ ^ - -L - ^ ' 

techniques for orienting supervisors and partfcipants to the programs. 

-Counselors saw the participants as being satisfied with the programs 
and as learning good work habits frCm their participation in the., 
program.' However, a sizeable number of counsifrk>rs«did not feel. that 
Specific specialized job skilH ^ereH)eing enhanced by the {program. 
Numerous program strengths were cited by fcounselors with a particular 
emphasis on learning good work habits and /learning while earning, 
^(ey ^problems noted were* insufficient morses, isolation from the job 
market, declining numbers of jort^, tDo few jpb sites, ana finding 
panticxpants lobs after they completed training. These results take 
^^.gi^ater meaning when analyzed in the context of supervisor and 

^participant comments. (Especially note the discussion of^ Table 12*: 
Summary of Program Strengths and Problems.) 

In Tabirr, the perceptions irf^ork experience supervisors are 
summarized. The supervisors were involved in the^ three most commonly 
observed CETA ;)rograms and the CSA funded programs. By and larger 
the supervisors ^had extensive pertinent background experience for 
working in the programs. Most of them had been orientedvby the coun- 
selors' to the programs/and considered the orientation to be adequate 
for their needs. They emphasized that participants were: gaining 
an under st anding of mrk ethics; learning coping/or survival sJ<ills; . 
and in ni^ instances, getting ^ecific skill training.. 



Counselor 

Experience 

— r 

College 
training 

(8) 

n 

PrdCtkal 
experience 
2 years or^ 
Riore (3) 

Practical 
experience * 
1 year or 
less (1) 



Counseling 

Caseload 

Perceptions 

Can dp a 
good 'job 
(11) 

Spread too 
thin (1) 



Summary of 
Nuflt>ers in ( 



SupervlsW 
Orientation 

Site^^itA 
w1 th partici- 
pants (3) 

Discussions . 
with Super- 
visors about 
Jobs, partici- 
pants and 
programs (8) 

Orientation 
booklet^/ 
materials (1) 



- ^ ' tablp 8 
Counselor Percej^tions of Youth Training Programs as Derived fr^\ 



Skill 
Enhancj 



ent** 



Sk^fl^ are being 
enhanced (4) 

Skilli not being 
enhar/ced 0\ 

Lear/iing good 
worl^ habits (6) 



the Counselor 

) Indi^cate the Counselors who Responded in a Specific. Way Out of a Total of 12 

Part)cipa))t . 
Orientation*^ 

Intake inter- 
views (3) 

Orientation to 
Job /esponsi- 
bititles. re- , 
quirementS't and 
pay (1) . 

Site visit with 
participants (1) 

Orientation^ 
booklet Wr 

Orientation «to 
Jobt responsi- 
bilities, dress, 
behavior, etc* 
(7), 

Group counseling 
sessions durinq 
orientation (2) 

Done on-the-job by 
supervisors (1 ) 



Participant 

Satisfaction** 

" — # ' 

Generally 
S'atisfied (9) 

Some complaints 
about pay (2) 

Some complaints 
about referral 
run arounds (1) 

Vbt satisfied due 
to lack of 
permanent placement 
and goal not 
be^ing met (1) 

Some problems with 
supportive services 

(1) 



Problems 
with Forms** 

Time 

consuming (4) ^ 

Confusing (2) 

Overlapping to 
a certain degree 
(4) 

CETA forms are 
difficult to , 
complete (2) 

No peal 
problems 

{5[ ^ ' ^ 

Too many 
forttts (3) • 



Interviews 
Counselors.* 



Strengths 

'Benefits to 
the community 

(1) • 

Ce^rning good 
work habits ' 
(3) 

Leamina while 
earning^ (6) 

Sqpportive 
services (1) 



Professional 
level of 
staff (1) 

Better self 
image on 
part of 
participants 
(I) 



Proble^*'^ 

Monies <3) ' 

Isolated from 
labor nyirket (3) 



Inif^ 



DecliniT^. Jobs (3) 

Helping participants 
to get Jobs before 
leaving Ii4;gh school 
oV afte^HP/ogram (3) _ 

Need more -Job si'tes ('2) 

No problems (1) 

Changing employer !o 
attitudes (/ree * 
labor) and parti- 
cipant attitudes, 
free Welfare, 
nipney) (1) 

More timeV.especially 
for teenager in the 
program (>) 



* One of jXe Interviews included in this set Is that of a counselor supervisor. 
** Several counselors mentioned moVe than one answer. 



Orientation 'of 
supervisors (pI) 

Iqq much time on 
paperwork and too 
little time on 
counseling (1) 
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Table 9 



\ 



^> Suninaty of .Supervisor Perceptions of* Youth Training Programs as Derived from Supervisor Int^rvjews 
' . Numbers In ( ) Indicate the Supervisors Who Responded Out of^a Total af 14 Supervisors , 
/ ' . ^ ^ • • • 

Psrst Pro.gram (Orientation to Ajiequacy of ^ Skill J " 
Experience the Program Orientation Enhancement 



• 2 'years ^r 
.more {9) ^ 

'^Less than' 
; 2 years (5) 



Vio grientation ft was Understanding 
for .supervisor adequate (10) of worK ethics/ 
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(1) 

Prior/similar 
-types of V 
programs (3) . 

Cqunselorjf 
agency 
drientation 
(10) ^ 



Not adequ 
<3) 



Adequate but 
others will 
*need more (1) 



cpping .skills 
(13) 

Skiir 

.training (^) 



Stfen^ hs"^ 

Income (^) ' 

The training*/ 
leading to a 

job'i4) 



Problems 

t Training not 
meaningful (vl) 

Nb job;- 
slots (U 



Getting "kids" NoVob'lems (2) 
off the 

streets *(1 ) Financial. 

problems (>) • , 

Cbunselor/ ^ 

participantw* ^- Participants/ 
rTPp^)ort (1) supervisors 
• ^ , heed more 

information or 
courtseMng (3)^ 



Suggestiotis . . 

Worksites should • ^ 
ptck up enrallees ^ 

Improve evaluation 
procedures - more 
feedback to parti- 
cipants and . \ ' 
supervisors ^(2)^ ' 

More*, counseling, 
testing, and • 
participant 
^Information (4) 



CO 

o 
I 



/ 



Profes^sional 
support of 
CAA (4) 



Low cost ' • 
labor in ' 
exchange; for 
training (1) 




. Participants . ^ 
, are not free 
:iab(|r (1) / 

Poor , 
partjctlJants 
motivation (6> 

Pro^rjims are 
too loose (2^ 

Soine students/ 

kids don't like 

^this particular 

work 1(1) ; " ' . 

Parents take ' • 
^ away eafning^s (T) 



Wore training (Z\ 
More money^ (37 • 
Mor* gSb sites (if 
Expand 

progrjims (2) • 

Provide a way • . 
for yOXPttifLfl 
offenders to 'get 
a work recordiH) 



Training, income, *and the, profess ioijal support of , the a§^y were .the 
molt frequently mentioned strengths of the programs, fogr participant 
motivatfon, the -need for more information and counseling, and , the lack 
of program structure were the^-most common. problems cited. Six dif- 
ferent suggestions for ^program 'improvement were recommended by at ^ . 
^^st two 012 more stapervisors. These are incorporated into the sum- 
m&ry of program strength's and.problems. («Table 12). 

In Table 10, the perceptions of the program participants are 
.described! Cl.ose to one" half of the partic,i pants w^re in the program 
for six: months or;mof*6^at the time^ of the interview wi^ the Vest of 
the participants in the program for a shorter period of time. Most 
of the participants^ had been provided an orientation to the pntaram 
anti. to their specific "^'ob duties by an ^ncy counselor. All panMcipants 
were satisfied witfx "their Jobs and the general nature of the learning 
experience- 1». a similar vein, they i^ndicatpd 'the aspects of the 
program they liked most we re/*the »1 earning experiences and the pay. 
'ElevjBn (5f the 17,-participaats felt that Vhere were no real problems 
withvthe -program. Others, although in the millprit^, mentioned 
.transportation, pay, and Tnore hours as diff^lties. At least five 
major suggestions wei»e-expresse(i^by a Yiumber of participant?. * 
These suggesiriofis relate to more money, more h«<jrs, more program 

structure, pdprly moti-vated participants, anH thfe obtaining of jobs 

^ ' , ♦ ' >* • 

upon comgletiori^of the^ program ,(see Table 12). 

In Table 11, a number- of .unfque in school and but-of-school 
programs as well^ias oth^ innovative ideas is suramarizecC. The table 
entries are based, upon the on-site ihtervieiSfs with agertcy administra- 
tioft aw^the perceptions of, the ijiterv1e\*ers- ^ ^ ^ ^ 



Table 1p 



,Sunmary pf t>artici pant Perceptions of Programs as Derived from Participant .Interviews 
.^NumberS in ( ) Indi'c,ate Participants who Respon(Jed in a Specific Way Out of a Tot^ of 17 Participants 



Length 

of Time Orientatioti to 
in Prftgr^m the Prograjr 



Satisfaction* Likes Most* 



Like Lea'st 
(Problems)* " 



Less than 
1/2 year 
(9) 

^re than 
.1/2 year 
(4) • . 

MoreHhan 



Counselor ex- Satisfied with 

plained pay,; job, learning, 

responsibilities, etc. (17) 
dress, etc. (12) 



No prot^lem {}\)r 
:Transpartatiot 



1 year (4) (1) 



3oiiiewhat but 
primarily 
dependent upon 
job supervisor 



Not far ^from 
home (2) / 

^Transportation 
The experience \ difficulties (2) 

CCAA helps, whole and the people 
family (1) (12) More hours 



^Somevhat But ^ 
not in detail 
(2). 

Via agency group 
orientation (2) 



See the 
counselor 
oft^n (1) 

Do not see 
the counselor 
often (1)^ 



Like the pay 
(4). 



needed (2) 
More pay (2) 



Openness^ df Too busy to 
jpb sftustidn really .get ^tra^ed^ 



on the job (1.) 
?*At^bme sites, 



Suggestions* 
More money (6) 

No SMggestifins (4)' ' 

« ^ 
More hours (6) 

•% 

More opportunity for 
more individuals to^ 
be Involved (1 ) 

More Structure to 
program (3) ' • ' 

^EnroUees should get job 
upon completion of 



large groups of CETA training (2) 
Workers caused 



^ . (rroblems fl ) 

- 'Personalities 
* ion^lh'fe Job (2) 



* Some participants gave more than one response ih a ques 



Expand it into more fields 
of study (1) 

Some enroll ees not too 
well motivated {2) 



Table- n \ • ' /. • 

Innovative.'Progfams* and Ideas as.Derived Mrom Agency- Inte^ 

In-School Programs ^ / Out-of -School PVograms ' Other Ideas 



Police Cadets 

Summer Work Camp 

Weekend Work Camp 

New Careers 

l^eCreationpl (Semi- 
Educational-)' Programs 

Job Corps 



Agency Run Small 
Businesses • - 

Experimental Private 
Enteirprise Programs 

Specialized Youth 
Offender Programs 



\ 



Periodic Group - 
Orientation and • 
Meetings for Super- 
visor^^ . 

Designated Program 
Coordinators in 
SchooH-^ 

Initial Group 

Ori en tart ion to Programs 

for Participants 



♦'Generally innovative programs were being carried'out on a. small 
scale: ' / * ' 



Table 12 represents a summary of program- strengths and problems 

9 

as determined across counselors, supervisors, and participants. The 
strengths as outlined in the table are a ra^ther clear indication that 
'in many areas the agencies have developed positive programs that are 
h^ving'an impact on participants and work-sit6 supervisors. Of , 
particular note are the facts that: participants seem satisfied, 
with the program; it is desirable to learn while earning; skill*.^^ 
and positive ,work habits are being enhanc^^and that there is some 
experimetitation with new programs. 

On the negative or problem side, however,' a number of -major 
diffic'i)lties are cited^ Some such as. more pay, more Tiours, trans- / 
portatlon problems, and a decline in the nuirt)er of jobs reflect a 
set of circumstances beyond the control ^of the Community Action 
Agencies; but others are" not. For e;camp]e, the fact that the forms 
are paperwork aragtime consuming, overlapping, etc. Is a difficulty 
vthat can be pv^pcome by a small investment of time and money. 
(Even if stand^irdi state forms are involved certainly changes can be 
suggested to -appropriate state officials.) 

-Five very serious program problems are identified. in the table. 
The' need for fr^Ve job sit5*and more training are Important areas for 
Jmpro^ement. . Getting participants jobs after they complete the prognam 
r^^ould^seem to indicate -the lack of adequate placement and follow^ 
through services. Poor participant motivation must be carefully 
examined inasmuch as this type of facjtor can greatly affect the success 
or failure ojF programs. The next problem is one that gete at the* 
^very core 0/ agency purposes. There is an apparent need to strengthen 
* agency counseling endeavors. Information dispersal and evaluation 
activities. • 
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^< Table 12 

Summary -of Program Strengths and Problems as Derived From Tables 7-10 ' 
•Source (s) for Summary Are Listed in Parenthfeses 



Strengths 



Problems ^ 



Some experimentation with 
new programs '(Agency • 
Administrators) 

Counselors are college trained - 
or have extensive experience 
(Counselors) 

A variety of •supervisor/ 
participant prjentation methods 
were used by counselors 
(Counselors, Participants) 

Participants generally were 
safisfied with the program 
(Counselors and Participants) 

Skills and positive work hab'its 
are being enhanced (Counselors, 
Supervisors, Participants) 

Learriing while earning 
(Counselors, Supervisors, 
Participants) 

Supervisors have extensive, 
experience with the pi^gram 
(Almost 2/3 of the Supervisors) 

Adequacy of agency supervj-sor 
orientation (Supervisors) 

Prdjfessional support of CAA 
(Supervisors) 



Lfllftted experimentation with new 
programs (Agency Administrators) • 

Forms'are tinte consuming, over- 
lappina, confusing, and so forth ^ 
(Courfselors) 

Skills not being enhanced 
(Counselors) ^ v 

Monies are insufficient (Counselors, 
Supervisors ,^artici pants) 

More hours (Participants) 



ipb market 



Isolated^n 
(Counselors) 



Decline in number^ of jobs 
(Counselors) 

^Need more job sites (Counsejors, 
Supervisors) 

^Getting participants jobs after 
"they complete; the program 

(Counselors, Supervisors, 

Participants) 

Poor participant motivation 
(Supervisors-, Participants) 



Participants and sypervisors need ^ 

more information, counseling and/or 
better evaluation techniques 

(Supervisors) !^ 

More training (Supervisors) 

Transportation difficulties 
(Participants) ' , 

<» 

More program structure 
(Participants) 
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(This need is denved from, supervisors comments regariJing the need 

• """"^ - < 

for more counrseling, the programs being too loose and the supervisors 

* need for more information as well as similar comments from participants 
thetnselves. -Overall, 16 comments were noted In this area from 17 
participants and 14 supervisors.) Lastly, experimentation has been 
noted as both a strength* ajid a weakness^ It is a weakness in *he sense 
that although all agencies are experimehting with new programs, they- * 
are only doing so on a-limited and infrequent basis. With regard . 

to experimentationKthe agencies note that federal rules and reflula- 
ttons are too confining and preclude the desirable tryout of innovative/ 
^ new prograifls. Suggestions regarding ways af overcoming these limita- ^ 
tibns will be offered in the last chapter ^f this report. 



Evaluation - Results and Discussion 

The fbur major areas for study in this project were: context, 
'identification, intervention and evaluation. The evaluation results 
are summarized in Table 13. The data that were utilized in constructing 
the table came from interviews with agency administrators. As obvious 
from the table, a varjiety of evaluation techni-ques is being employed 
by the six agencies. Ahd, aside from the on-site evaluations of work- 
sites and the keeping of records, there is no clear cut agreeme^nt across 
^ the agencies with regard to evaluation procedures^. A number of agericies 
» indicated jthat they were just in the process of. starting their follow- 
up studies and^did not presently have data from those proceijures. 
(j^ote: one agency did have an external evalwator review its out-of- 
%chool program and prepare an evaluation report for its use.) 
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Table 13 

• Program Evaluation Siw^ary as Derived from Agency Interviews 
Numbers in ( ) Indicate the Agencies that 
Responded Out 'Of a Total of Six Agencies • 



Evaluation Techniques* 



Program Success* 



On site evaluations of 
worksites (40 

Randojfn follpw-up studies up 
to 1 year (2) 

Word of mouth feedback (2) 

Follow-up studies up to 
go^days (2) 



Input measures (such as demo- , 
grapliic information) (3) 

State field representatives or 
-monthly- reports (3) 
'* 

Student self evaluations (1) 

Records, posit<ve 
termination* rat-es (5) 

Limited -or no follow- up (2) 



Evaluation still in process (1) 

Program seems successful (but it 
provides mostly pay and little 
training) (1) . • 

Program seems successful (5) 

Difficult to defipe positive, 
criteria of success ^nd to deal 
with turnover rates** (3) 



** Some agencies gave more than one response. 

** This was the response gv/en by three sites in answer to the question 
about program success. / . " 



^ In terms of program succejfs, most of the agencies felt that they, 
were successful. Not much data as indicated^above were available, 

Kowever, to support the perceptions of the agencies. One problem 

t • * 

noted by three of the agencies was- the/difficulty of establishing 

eriteria'for determining program success. They cited such factors as 

high turnover rates, ^program- completers leaving the area, the cost of 

r , ! 

evaluation, and so forth. In one case, an'agency suggested that it ^ 
was especially difficult to establish criteria for in-school programs 
'which'have an implicit goal, of "keeping kids off the streets." In 
this case, for exampTa,\js a sligHt reduction in the dropout rate an 
indication of program success? , 

•The strengths and problems with regard to evaluation are very 
simply described and almost self apparent. The agencies almost 
uniformly expressed a need to imprave in the area, of eva^luation, 
data collection, and so ferth. They were acutely aware of short- 
comings in this area. Their awareness is considered to be a strengtli. 
The problems lie fn-the lack of actual \data collected arts' in thejjeed 
for systematic da1;a collection procedures. One other problem observed 
by the, interviewers is that the progyfims are often thoughts only 
in ter^ns of positive placemen«t of participants. If .sub-goap in the 
areas of attitude change, skill derot^prafint, coping behaviors, etc. • . 
were assessed a different picturV^f^sults might emerge. - 



V-. RECOMMENDATIONS 
The recommendations in this chapter are based upon observations 
noted earlier in the report ^nd* on the professional judgments'of the 
two interviewers. They represent suggestions that, hopefully, if even 
partially implemented, will lead to improved agency and program 
operation-. They are made in the spirit that inany aspects of agency 
operation are well thought out and implemented but that others will 
need improvement. The recommendatiorvs .are organized in accord with*' 

. the following areas: context, identification, intervention and 
evaluation. It should also be noted' that the recomrriendatTons are 
based on a case study approach which involved a small sampT? of 
agencies and -a relatively small number of site visft? ajid interviews. 

^ Large scale sampling may have revealed a different set of findings and 
potentially different recomrondations. 

Context Recommendations 

Inter-aqency linkages, i.e., commuriogat ion channels, .should 
be improved . This problem was most ri^iiceable in two of the 
> 'six comitiunity action agencies cited. Better coordination of 

programs hopefully will, in turn, lead t<^ better programs for 
potential participants. The communtcation recommendation 
really has two parts. On the local lever, area-wide committees 
of local social service agencies should be formed .or ,>if 
already formed, should be maintained and strengthened. 
On the state level, the DECD should fnake every effort^ to 
improve the. coordination between its agencies and-^gencies 
like the Ohio Bureau of Employment Services | (OBESj . - ^ — ^ • 
Although not expressly noted before severat local agencies 
identified this as ^ linkage problem. State coordination 
would probably help to alleviate the local problem. 
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2. The DECD and local agencies s.houl 4- carefully monitor the 
dispersion/allocation of CETA monies . Tfi'ere is a ly^elihood 
that CETA-funds may be shifting to local units of government.. 
This tr^d, if it indeed is one, is^ftot necessarily negative. - 
What is important here is ttot so much that .one agency or ^ 
irt^titution has the funding in place of another but .rather, 
whether the agency that can do the best job of training is 

* actually receiving the monies. This is a most sejrious concern; 
on both the local and state level. . ' 

- w 

3. " Reconmernlations in ttie area of transportation will be -found - 

under the Intervention .heading^ ' ^ 



Identification Hacghmendations^ * • ^ ' 

As noted earlier in the report ;_the communication and identifica- 
tion procedures of the agencies seem to be operating fa^.irly well. 
These recommendations therefore are offered for the purpose of 
improvijfi5 already proficient techniques, 



• 



. target some publicfty efM^s toward 9 wide range of prospec- 

tive employers especial l.^n the metropolitjin/urban areas . 
^ Handbills, procedures, fliers, pamphlets are typical devices, 
that 153 ght be utilized. A wider spread of training sites 
^ f}^ight 'th^s be obtained. 

2. Focus greater attention on specialized subrgrotfps within the 
' disadvantaged target populations . Recent federal program 

emphases have been in the areas of: women, handicapped, rural, 
youth, bilingual, etc. * (See Employment and Training Ad^ 
ministration programs). Some of these groups have sfpecialized 
types of problems (e.g., the isolation of rural jwomen, day- 
care, needs of young mothers) that require a unique or different 
' • type of program. There is a strt)ng possibility th|;t focused/ 
targeted programs will become more prcvninent in the near 
.future. ' * ' • ' 



Intervention (Program) Recommendations 

Experimentation with new*proqrams should be expanded . - 
New training settings both public and private should t)e explored 
Alternatives to existing counseling, follow- through and rating ; 
proco du ros ecu 1 4-^ tried. Different modes of supervisor- 
counselor interaction could be examined. 
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Better techniques foh monitoring partioipant Ijehavior could 
i be (feveloped. Alternative methods for funding programs and/or 
Paying participants might be studied. (In one agency, for 
. example, some counselors spend, up to tvtq day$ a week just 
delivering pay chedks). . • • . '• : ^ 

' ' ' « ' . * * " 

Regulations and rulfes may be a deterrent lo. trying out new' 
--and diffeWnt prograns. Frequently, however, the federal i 
rules can^be'' relaxed to allow fon experimentation with * 
promising new ideas. Allconcerned agencies are encouraged, 
to see if-tfleir ideas can be implemented with full governmental \ ' " 
, approval. . . . 0 \ . ' *' 




^fH Forms > especially CETA forms could be improved . - Several of 
the acncies reported difficulty 1n completing CETA forms or 
in getting certain CETA fi)rms completed by potential 
participants orvtheir parents. These difficulties are treated 
at the state and federal, level where the forms originate and 
corrective action' can easily be taken. (One option* might be 
. to hi.re.a consultant at -a nominal cost to revise and con- 

. • SQlidate the forms). t 

j , - . ' , ^ 

Locally made instruments could also be improved by tfie use- 
of better scaling techniques that are based *on obsfirvdbl.6 . 
ehaviors. (Also see evaluation recommendations^) * . \ 

Identify more varied job sites. with better training possibilities 
and possible long teijn job openings . This, recommehdat ion has^.^ 
'already been covered 'by recommendation 1 above an4 by the 
first recoramendatiop.in the Identification section of the 
/-report. • , - • 

^ Part^ticipant motivation, to thg extent possible, has to be 

improved . A's stated now, this is an impossible recommendartton 
* to implement. Yet at the same, it is at the very heart of the 
train fng/work experience programs. If participants are not. 
motiva^ted they may in turn "sour" or "turn off" supervisors to 
-further infolyement Jn the program: The next recommendation 
* .'^eals with perhaps a partial solution to tfie dilenma. 

5. Strengthen the counseling functionsMn agency programs . ^ 
The counselors at most agencies are either well educated ^ 
a^d/or have experiential background which'qualify them for - 
itheir. positions*. Ilowever, this seems only to be am adequate , - 
set of initial skills. Counselors could benefit; from additional 
training in: . reinforcement techniques; group/individuaT ^ , * • * 
counseling techniques ;%)ti National strategies; techniques for 
evaluating performance and -use's of evaluation »data; etc. 
Counselor training probrams we*nd'seem to have value and • ^ 
should be explored. ' The goal/ti^re is not necessarily tp * . 

• ' get uniform counseling 'procedures but to get better informed, 
better traineid tounsielors.- • , 
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Etefine or specify in mo r^e detail the vole pf the supervison ' 
and^anti'ciparlt . TlvtV rebommendation js partly an Outgrowth 
ofj^fi^ ne§d. for-a stronger counseWfig tij/ction. . Somes 
'su;5ervi Slot's, and' parti cj"gants^een).! to m^d clarification of * 
their rcfles-^n'd fesijonSibilitiear- Contracting through work 
'agreements iind the Qtilization' of specialized program^* ^ 
orient^tiort sessions ml-ght.help' to>. alleviate this.probl^iru ' 

^yisB alternative phaqrams inrura-l areas that Wight-help 
. to reduce tran«Dorta.tion-j3robleras . . This mi-aht besarxDmol 



.7. 



ished 



/ to rejduce tran^portauti on* problems . , This mi-ght besauxiinipTi 
'^y settiBjg up special weekfend work progriris for in-sch^l 
parti cip^nts-and then^se^ng conwurfity. volunteers who would 
b'e wiUinfl to^roy/ide bfarfspbrlaH small groups.* Mhfe 



programs* could b/'Tn .tH^form 
home. Winterizat'ioHj 'and camp si 




nity beai^tifi cation, 
Jopment p^jects.' ^Jhe 



, 0* 



r ♦ 



^\ provis^iorv of ^tratlsportatidn -could^pe viewed 'as a charitable*^ 
• Ka< deductable) dSMIt to a non-spKofil? institution; j 

8. ; Develop -programs f6r the irir^Choal CETA prograifc tfiat do not ^ 
- opgrajbe 1f1 jd^jge^^ t . Recommegdatton seven ts an example of this 

'/^^ type of program.: This recommendati^, whic^is based partly 
upon ttie intuttion of the intervi ewers j.^stemS'from the observa- 
tion thitaittjjy CETA enroll<ees may be marginal Tfi ferms of 
school TnvolYemet>t and may -have negative perc^ti*qns oi the* * 
schooj . By -ma^taininV literally "in^schoo]" 'programs these 
perceptfons (ft^f'^be neiRf^^ ^ ^ ^' ^v- ^' ^ 



^^ Eval»ue>tion Recommendations 




> With state assistance, a set of procjgdares ,and guidelines for 
^nral A yaluation'jof% programs *shOirld be^ developed : ^ Agency ^ 
TKTuatiaa* in dtisarray. -Forms are'not cdn^is- ' 

tent acr^^ agencies and often are hot of high enough quality 
♦to jK)vlde»usefpV infbrfnatioa; ^ Evaluation is not dUiigh • 
^ pf^fPnty for>ocal agencies. Supervisors, in some instances;^ 
. do not see tne Val^^ of carefully' evaluating participant 
performance, , ' 

. x 'i . ' ^ ' ' * . 

Jhus, the Siigggsji^that-trained. e valuators at the stat^ 
^e\?Sl develop iyith J deal input a set of forms and procedures. 
'Ihe forms/profcedures' should be strandardized and general izable, 
'^,but at| the same timer |hsurftr provide. .for the use of local ' 
options and" variations. This/W^ld. require that program' 
•objectives be defined* and potenlially that programs couJ.dT)e 
evaluatecTiagafnst a wide ^s^t of criteria in addition' to solely 
■ the .criterion o^F placemen^.. Statewide evalJuationUechnique * 
workshops sliould algo he f^d. (Consultant help and/on^the ^ 
. creation'of -a Statewide tvaitfation positipn should be 
' conypltred.) , * • *T * ' , 
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[ 2; With state'assi Stances^ set (jf procedures for conducting 

• regionaj fpllow-up studies on a sampliag basis s>ipuM be " , _ 
^ ' / developed and jifnplegiented^ / There is'jio clearcut evidence ' ' 
of program effectiveness. It 1^ doubtful that continued long* 
tei^m funding can be expected without such evidence (see lacail 
^ variations above). * . . {* 
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Introduction . \ \' ' 

The purpose of this' appendix is to-'briWfly^^dis\auss some of the 
bafckground factors related to the methodology used for the stud^^ 
described, earlier in this report. The treatment of thes.e-^tors is 
not intended ^to be exhaustive but rather to provide the r^ji^r with, a 
surface L|mi|^stand1ng of sdme of the^ considerations that preceded 
the selection of a sfeclfi^R&tlfodologyi. -Many issues (pros and^ cons) > 
reTated to variou^- methodological apprdacHes are. al^ included ^ ^ 
this appendix. , ^ 

. ^efal Background factors ' ■ ' '\ . 

In an age of accountability, concern about the evaluation of 
community action programs has been very evKlafJl..; For example, . 

. Anderson and Whittei) in a recent ( August T976) newspaper article 
.noted «that " . . apparently no^ one in .Washington knows whether the 

* government's anti-poverty programs are doipg any good." They further 
, state that approxima'tely $1.5 billion i-S^ spent irt antvpoverty programs 
by state and local. /iSids. They conclude by re[)j)rtlag that a recent;, 
confidential GAO study indicated that less than half of the required 
progress reports from local programs were reaching Washingtvi .and that 
many of those reports were inadegudte-*(Anderson and Whitten, 1976) 

Tha government as well as individuals who work in the anti- 
poverty field clearly recognize the need to be accountable and to* 
definitively evaluate pro ^rams.^-' This need'is prompt^by two major 

s underlying causes: (1) where larg^'sums of mon^^ are eiqpended, the^ 
public and thfe legislative bodies wiljl over time demand that the 
Yecipient agencies be accountable for the effective ftcogralfHHtlc use ' 
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• - 

of funds;- and (2) the collection and use of evaluative data are 
^ • "I 

major ways of refining, improving and further developing programs. 

This is reflected fn the description of the CAA planninglprocess, * 

♦ 

which includes steps for -monitoring^progress and evaluating effective-. 

ness, and in the budgeting categories .!ff the^ffice'of Economic 

Opportunity.* Jhe latter contain specific and separate .budget categories 

for CAA evaluation; Manpower Pro*gr,ani Intake, Assessment. and Program 

Placemeat; Headstart Evaluation, and so on. * . 

WMTe the need for evaluation is clear-- how to ^evaluate is not > 

• One suspects that the GAO observations are not oijnyTrccurate 'but that 

^ • , • • • 

they arre so for two disparate reasons. First, the, recognitton that 
evaluation is necessary and* useful is probably'* more firmly hetd bjPthe 




hi gher/ad)^ni strati ve levels of thV^system than'By the lower levels. 

And secondly, evaluation of comple>^proces$es and outcgmes in realistic 

life settfngs is methodologicall^and practically difficult. 

In a r'ecent study in the related field .of juvenile delinquency, •* 

conducted ty The cAter for Vocational Education, the -following kinds 

Vaf observations were made: . V^. , . 

* . 

•1. Systematic program evaluations were alofwst 'never conducted; - 

• "* ♦ 

. Mortality, i.e.; drop off in sample, size as a treatment, or 
^ ""pr^gram is in. process, was rarely taicenj'nto accountj, 

*3. Feedback on individuals who had beeo'Yef erred to other 
, agencies artd services was generally not, obtained? 

4. Inter and i^^^ agency communication wa§ often lacking or 
was simply nn^jjfnal and loose knit'; and . 

5. "Many project ^staff beg the evaluation question by. claiming ' 
that one cannot adequately measi^re subjective behavior r * . 

, (jittitudes, etc. .and .-lAat there is not the expertise ^ai table . 
to execute adequate evaJuation." (Cardfrelli, 1976)^ ' * 



59 



(It should be noted that the" report also does cite •examples of reason^ ' 
able or adequate evaluation studies in this areaj These observations ' 

^tend to support the tv^ assumptions made above, and to an extent 
probably are general i-z&ble to that .of youth, oriented training programs 
irfOhio.'s comnuDity action ac|encies. - ^ ' * - * 

* "•^Several- important distinctions, however, must be pointed out. 
Frequently, the parameters, of educational types of progirarfls * though 
not furll^ defined are05W:ter defined than thosfe of juvenile delinquency . 
prevention projects. Objectives in. clear terms may. ^be' available or , • 
inferable, average duration of the tre*atment will be specified and • 
so forth.' Secondly, incMcants of success, especially in performance'^/ 

. areas 5r are jor can be de^Velpp^ f^j^ use in assessing program a^ffeCtiye.- 
ness. These indicants would obtain a higher degree of acceptance by 
staffs of the commonity actiorjprgencies than'wou^ld analogous ind^qants* 
in the juvenile delinquency field." Lastly, one ^qf the major outcomes 

of th>s particular evaluation of cofnmunity itction .agencies.fjor the «. 

Jr ' . ' . ■ , , ^ / 

hio i;S the idejitifi cation of program strengths and weaknesses *^ 

accompanied b/ a strate^ br, procedure for assisting 'local agencies 

to improve their lef&l of performance. This assistance sholjldtend 

» . * ^ '^ 

to cast the ^evaluation in "a different ligb*" a positive light— 

rather than having an investigatory connotation. i ' • * ♦ 

The Methodological Approach —A' CI ostr Look 

The question still remairfs, however, "how should yout^ orifented^ ^. 
training programs in'community action agencies be evaluated."*^ Useful/ 
insights irtto this question are found in Evaluating Action Programs : 
Readings in Social Action and Education JWeiss , 1972)" In the first 

1 



chapter^ "A Treeful of Owls," Weiss presents an extensive discussion 
of i:he evaluation "of social action/education programs. She ,notjs that 
m^ny evjaluatjons ^sist of an individaal or a team gotng to an agency- 
and asking questions. This "impressionistic inquiry" relies heavily, 
orvwhat people are willing to tell you and probably is not as objective 
as one would like. She continues by, stating that often questionnaires 
$re used as an ev^aluition technique because they seem to be more * 
objective or scientific. Bu^ this appnj^ic^ Having sonje 
advantages, e.g., lack of invol ve^febt^of an interviewer with the / 
interviewer'^ associated biases, yieldirfg Qf clues •fegardihg program 
strengths and weaknesses, does^ave major disadvantages.* They are: 
respondents only tell yoa'what they Want you to know; generally the' 
number of questions, is limited;* the depth of (Questions is Tirtifed^r- 
and so on. ^ • • ^ ^ . 

Weiss concludes by stating that eventually evaluation comes igp 
the basic question of how w^lT is the program, ac^ompl ishing what it 
sets out to accomplish* For evaluation purposes, this rqt^uires that 
Is of ? program are carefully defined and that they can be 
an^lat^d into measurable indicators o1^ achievement. An;) further 
that the extent of goal achievement rcan be assessed by use .of ex- . 
periinental and control (or equivalent) groups. Weiss, though an 
advocate of evaluation against goal criteria, is not naive about the- 
difficulties' of implementing, this approach in the^real world situations 
For example: ,goals are often neither simple nor clear cut; control/ 
equivalent groups may be difficult, to obtain;, evaluation is a secondary 
activity as compared to the primary purposes of agencies and/or 
programs; program staff may not be 'too wflling to cooper*alte with 




evaluators; programs, themselves, aire not" always sharply -deffltd! 
entities but ra'ther may be more ^amor-phous in form; ttf^ process of ' 
determin^ing if a program has achieved its goals can only be ac- ^ 
complished after the program is completed and hence the immedTate^ 
utilijty of the results is reduced^ and in general, evaluation has not 

that much iiT^)aqt on the ;decisions/decisiort making process. 

Cohen in "Politics and Research: Evaluation of Social Action 

Programs in Education" notes that in the mid 1960''s jwe witnessed'the 

f 

estabVP^hment of wide scal^ educational improvement programs funded 
by federal and state sources. (Cohen, 1970) These programs were w 
reality, ^ocfaj^action programs aimed at broadly, jnproving educjation 
for disadvantaged students. The pVograms are characterized as being 
politically conceived,; operatingMn multiple^^Jpied settings; and being 
difficult to define in terms of goals, treatmentCs) , and criteria f or ^ 
success. This; in turn, has^ resulted in poor or inadequate pro.gram 
evaluations.' ^ , . 

As an example, Cohen que^iotis the use. of achievement test scores 
as a criterion for assessing Title I (ESEA)' progratn aims—are th^* 
a comprehensive and representative sumMry measure for' adequately^ 

4 

assessing program success? He further postulates that the "Title I 

program' is* (not) sufficiently coherent and unified to warrant the ^ ^ 

application of any summary criterion" of success^ be it achievement 

or .something else." 

Coherr concludes by suggesting that: 

One purpose of program evaluation should be the identification^ 
* ' , delineation of program- ^oals (rather than -having the 'evaVuator 
simply lament th^,iac1^ of claifity of program intent); and 



^. Eva-liiating social action prcJ^rams requires comparatively 
^ . broad systems of social measurement including possibly a 
^ census type system of social indicators. ♦ , ' 

^hen*s thinking ^d analysis' are general izable to the evaluation. 

V of y^tf) training programs in colnmunity action agencies. Basically, 

saying that the evaluatioD must recQ^ize and fpcus on under- 

' standing ^he complex set of political social varia.bles' that underlie 

•progr^s. Programs aVe generally political entities vwth diverse, 

wide ranging goals. Program treatments will not (and perhaps.^-should 

notX be constant. ar static. Th^ will vary within the general parameters 

. of the Ifigislative/governmental mandate by site and according to the 



constituencies being served. Program^ evaluation must take in^^ account 

these 'primary cbnditioas and must be viewed from a broad, 0iot narrow, 
• — * • 

perspective V - * 

Another viewpoint that might be utilized in evaluating manpower 
.programs is ^ that of the economic effectiveness 9f programs. / However, 
"when* programs have objectives that go^ beyond simply maximizing the 
return on public investments ... a simple benefit— cost ratio js 
an insufficient* indicator of program outcome^|^(61ennan, 1969) Ac- • 
cording to Glehnan, an alternative would te to develop a system of 
weights which reflect societal values and which could be used ii 
^determining benefit-cost ratios. The difficulty w\th this approach 
is that it is difficult to measure or assess, societal values. And 
t even if this could be done the question still renins as to, how- 
.various societal values could be combined into a meaningful index 
• of benefits^. (Referring back, to the earlier discussion of Cohen's 
writings "the variables are a- complex set wRich inr tui^rf exists in a 
complex milieu Scpolitical and social factors.) Glehnan also 
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•indicates that the_data collected for "benefit-cost ^analysis are \ . 

frequently misleading and/or unreliat)le and often lead to confusing 

' * ■ ■ ■ . 

'benefit-cost ratios. He concludes his consideration of this type of 

analysis by noting that although one suitable method for evaluating . 

inanpower programs is by* the .use of the non-equfvalent control group 

method, . 'it too is fraught with methodological proBlems. * . ' ^ 

^^ome recapitulation is necessary at thisYflint. The question. 

' raised initially in this section was "How are.Vie.se manpower progHims • 

to,be evaluated?. Three different approaches weT^ discussed': 

An experimental" assessnfent^ of program against wel] established ; 
goals and objectives (Weisl) ^ . ^ 

- A brt)ad;based assessment of tHe'^complex milieu in which jfograms 
oDerate followed by comprehensive assfes<msnt. perhaps usTng__i v 
•ceVisus types of social indicators- (Q^n/r and 'i 

- Benefit-cJst analysis which may employ dKperimpntal types of 
techniques (Glennan). • ^ 

All of these have, a^ was indicated, serious deficiendtes. In ad- 

dition, the magnitude of using these -approaches' in this specific 

•instance is beyond the available time and dollar resources. So the. 

ihrtial question still remains. - \ ^ 

Peter Rossi in an intriguing, article entitled "Boobytraps and 

'Pitfalls in the Evaluation of Social Action Programs" has provided'. 

a partial conceptual answer to the question. (Rossi, 1966) There are 

' major obsJ^fes jn evaluating social^ action programs includixig: 

• programs^Hb work well on a smkll scale but cannot be generalized 

due. to a Wk of comm^fnt on the ^art of staff arid/or style and type 

' of |eadership^Jjj»|^/Tferent site; and the vested in^^rest of prograom 

•administrators If maintaining a program. He further suggests that 

evaluation be viewed' as a two phased process. 
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A reconnaissance Phase in which soft cocrielational . 
detigns are used to screen out those. programs it is 
worthwhile to investigate further; and an Experimental 
Phase in which powerful pontrol led experimental designs 
are'us^d to evaluate the differential effectiveness of a 
variety of programs which showed up as having sizeable 
effects in the first pha^.' - 

- . ; • ' 

Clearly,^ Rossi *s position, would be in support, of a reconnaissance 

ty-pe pf study for evaluating youth tt'aining programs in Ohio's com- 

munity action agencies. . * * . 
t ^ 

Additional support for this position can ba^und in the work of 
Weiss (R. S.) andJRei/i (igsg*). They contend that viable aUernatives 
to experimental methods for studying social .iction programs are: 
process oriented qualitative research; Iristprical research; and case 
study or comparative research. According to Weiss and- Rein, there are 
simply too 'many defici^encies in other methodologies. 

Methodology^-A Final ffojbe v ' . - 

Obviously , ^this methodological discussion could be considerably 
more extensive than it presently is. But even- this short discussjon- 
shbuld help^the reader to understand Ahe difficulties inherent in 
studying community action programs. ;, 

^ Based upon this brief review of the literature, a decision was 
.made to utilize a Phase One -Approach such as that suggested by Ross'i.^ 
In th'is study, ^the ^e'cific m^odology^s case study in nature and 
consists of two sets of interviews. The yirst intferviaw's involved 
agency administrators and the second Included aje^ider range of 
individuials 'involved in programs (e.g., counselors^ supervisors,"^ • , 
program'partici pants). The decision was based on the followiag 
©onsidffrations: 



■ , \ - • • • 

• y - 

1. The study was 9 first outside or external look at the 
programs of six diverse agencies; • 

2. Data sources (e.g.^, evaluation reports, records, etc.) 
* would vary exttu^fively from site to site. 

3. Progl^^^^^^^emsel,ves , are ?rot uhi form and will va,ry 



J. 

across sites; and 



' 4. A series of interviews per site -would- reveal or tend ^to, reveal 
those subtle factors which affect program success or'fa-ilure; 

It was recognized that 'this approach ?ould le^d to conclusions based • 

' upon the subjective jud|inent of practitioners in the held.^ Yet, at 

the same time', all attempts were made in the sTtudy td look for trends 

» 

across the six sites visited. It is highly unlikely that one would 
make a faulty or spujious conclusion, if the wtire set of sites 
provided similar information -in response to specific questions. 

Thus, while the methodology employed in this endeavor is not . 
withojjt deficiency 'or error, it is a suitable method, given the state 
of the* art in evaluation and the- nature of the programs being studied. 
, ^e reader of the report should carefully examine conclusions in 
light pf the-abbve methodological discQSSion. ' ; 
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Aqgncy Overview 



O. Please give a brief capsule des(iri{).tiQn of your agency 
-2. <^jBTjS(Chures^bli cations <i 

Context . ^ 
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Iv Please descri,be tbe, youth priented training prograife run by or 
y und^r the jwi^sdiction^f your agency.. , ^ 

2. Where § re these programs'.! ocated/hoxi^||^ ' * . 

3, - Do .you have- any brochures/publicatipns that describe these programs? 




Identlfltatiorf ' > - * ■ ' 

'~ — — ^ . ' - : . • " • 

1. How do potential cli^ntel-fe/Jearn of youth training. programs? 

2- . Hovr4p you' select peof31e to te involved in the xrainihg process? 
^3..= Are s^ individuals excluded/not involved^n tJre programs?.^** 

' ^ I' 

^. Whart is the canteflt of the trainjng progjaaifit ' ^ * • 

2. How long do the specifict prdgr^this lastf . * 



3.. Who teaches the^studefits? 
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4. .What types of 4nstruc?ti6nal in^terials aroused?'* • ^ v ' 

5. How is stlident ppcrgress eyal#ate^ -(Emplhasize the concept of 
student p/^pgressl) - . ; ^ ^ • ^ 

'Evaluation . . ' ^ , ' . • 

K How do;you7what techniques are<^you usin^ to evaluate your progra^? 
Z.-* How successful have your p/bgrams beeh?.^ " ' T ' ' ' ^ 
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. Overalj/^estio^ 

1. (Acrlssi eve rythtri'g we've talked about, please tell me what you 
\onJ^ider to be the strengths of •the^gV'aiTi and why. • ^ 

• 2... Pleas»»tell me what you feel are JjjLfrobVen| or><eaknesseS in . 
the program. ' i<hy? - 

If you hSh to'do thfs in depth evaluation, whit^ea would you 
focus on? ' . i " - ' . 
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Administrative IhtervieW Form for the Setbnd Site Visit 
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Introduct1oiiyi?.urposes ' 



Basic Stud^ltrateqy '* • 

' * .* - 

2 phases of Site Interview^: ^; ' * 

Initial Phasei Qyerviev/ and BxQloratioir • 

In-depth Ph^se: 'Closer moVe d^tail^ look.at yo«th . 

training m;^rams . v * ' • 



B. Beyond an AdmifTt^trative Lotplc 

• Go to individuals irivolved such as -counselors, enroflees/ - 
participants, supervisors , ^ ' . . i - 

C. P-urposes . . " , ^ 

1. .To e)(afnine in -grteater.dBpth specific/jihnovativa progra'ras 

. ideTitji(^'e(J on'thfe visit T ^ ' . . 

* ; r ^ ' ' / * ^ ' ' , . * 

2. Tp examine* in*.gr6ate'r depth some ^key are§^,reJated to, 

, ,^ youth training programs as' i^tified across the. s^x sites 

3. ''To1)btain an unders^ndir{g ^ Jeas^:' a' (yirtiaT j^ef of • 

how. program parti cipirfts perceive the program 

4; / Jo cohtfnue e^xploring, * , ' * 

D. Possible Outcomes from the Study ^ . ^/t' • 

1. Difficulty in devising'a. solwiion that is workabie across 
six- sites. ♦ . # 

2. The Sampler '^'dea 

. Innovative f)rograms- (8-)2j . * - • 

Suggestions for Improvi'n^^ operations - eva^luatiorr 

. - Orientation 

> ' ' \ • - information \ ' 



Agenda (Suggested and Ideal ' ? ♦ ^ 

A. Agenqy Wide Areas - \A 1?^2 hours ^ 

1. Initial Site, Visit Report, . / ' v 



Was the report acfcufate? Was it adequate "fn terms of 



ydurjjercep^tion?' 



bVlen'tatioh Process^'' . ' • ' 

What ways or techniques are, you using to orient tJie following 
individuals to the program?- Supervisors: 'matfrfals, • 
meetings,, (employment of minors), group exchange of, 
ideas? Enrollees; >pre-job orientation, meetings-,* 
materials, dress- ^havi or?, , . ■.' . T 

In terms of your work with the schools, how do you and' • 
the school interr£l ate/coordinate your efforts? .OWE? 
OWA? C&reer educatfont ^ Joint planning? ^ ' 

In terms of information flow, please describe "how you 
afld your enrollees, counselors, etc.. -obtain job/educational 
inforniation? OBES> 'Other.-agencies? 00 you irave any 
proUem^ (Sbtaining tnforn»tion? Do you systematically 
collect/use job information? ' , - i 

• In terms' of electing job sites,, p^lea^e d^scr^be the • 
fol^lowing. Ho\M6 emi^toyens 'l«am ofA^ep updated with . 
regard to your .program? How dp you evaTuate/dro;>»5ites 
from .further, inclusion? How; do you select sites to be 
.incTuded in the program?.^ jWhat criteria?) . , 

■-^In terms of evaluation, could y6u pl^se discuss the , 
following: Ca) proliferation of forme (stite-hnd-local , ^ 
(b) utility of^oi^Tis, and (c) -Cqllective/lon^itudinal;^ dat^: ■ 
increase in enroll.ee punctuality, attitudes, attendance, • , 
any "long term pl-Scement -dat*, . Please describe role af 
rpg-ipnal manpower .board'.' - . . ^ - 

'•Ari there afeas whet^A tf^Lthat you afljupli eating; 
services oc where" your, services a're'^eTiTg^'wrf^j.^ated . 
•Schools? 'ft'ES? Welfare?' Sta-te services? If soTHww . 

• (*ul^l this duplication- be eliminated so that you could , ... 
increase jeffectivene.ss/efficienc/? \ * ' . 
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Counselor Name 
Interviewer 



Counselor Interview - 3Q,Minutes % ^ 

T. What specific -prdgram(s') do ybu worlc with? .What is your background 
for this type. of joj)? • - ' \ 

2. What is your average. weekly/monthly caseToad? Question the adequacy 
of counsfiTing offered. ' - ' - 

3. -What techniques/ways do you u?e of orienting?. Supervisors to: 

sttfdent/erttelyed needs r enrol lee characterl-sti'cs/problems, ways 
to reinforce/Support^ enrol lees. Enrqllees to: job reqtitf^fewts, 
wfiat' to 'expect, dress, behavior. 




How do you' evaluate ^whejiher site (supervisor) • is ideally providing 
k meaningful v^nk 'experience for enrollees?. If not, what io yottilo? 

To what ^«f!j^ you feel that the* enrol lees are satisfied with: 
(a)';tHe couR^elirig, (b) the qoal-it^f supportive 

' seryices'^(ti^a^^fpo^^ referrWs), (c) the 

adequacy ^of pay, etc' / ^ ^ 

<0o you h^ve. any-^Orob^ ems with completing forms especially 
; evaluatipo formsc ''iime cgnsumin^? ' Overlapping? Clarity? 

/ > : ^- • / ' ' ' 

7% ^To what^ extent do you feel. that the job skills 'of the program 
" Vpnrollees 2^re really beifig enhanced?' Please explain your answer. 

8t Could you^ describe to me what you consider to be the best feature 
' 'of the pi^gram? Probe. , . ^00^ 

9. Wffef or^ ar^a of the program do y'bu think )vost ne6ds to be 
'* imprby^d?. • ^ ' , ^ - ^ , .'^ 



» ^ 

Interviewer 



Supervisor Interview - ZOl^inutes 
^ . - 

1. Which program(s) do you work with and imn many enroll ees do^you 
supervise? ^ ' \. 

2. How much experience <Jo y^u hgtve 4n workin^g with these progr?ims? 

• How did the community action agency orienl^you to the prograrrt?* ' 
Counselor visits? Visits ^from agency adfnf|istrators? 

4. Would you consider this orientation adequate in terms ♦of your 
understanding of enrollees, enrol lee needs,' ways of helping/ , 
reinforcVig ^rt^ollees, r' \ . ■ i ; 

5. What types of gob, training and/or jqb r^lated^kil Is do you feel 
that enrollees are gaining from working here. Please be specific 
and describe' how they gain these skills. _ 

6:' yQuW yQu describe the major >rdblem(s) you see;i^/ith the program? 
Please be frank. 

7. Wha^d(|you consider to be the most positive aspects of the program?- 
8.. What^suggestions do you have^for improv4i% tfie prc^am? , 



Thanks. 
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Enrollee Interview (15 Minutes fier Enrollee) 

Question 

1. How long have you been in this program? > 

'2. Before you started working here, did the 
CAA explain to you: . ♦ 

What would be expected oi you? 

- your responsibilities 

Your punctuality? - . - ^ 

- dress? ♦ 

- pay? 




3. Are you sttisfied with 

- your job duties 
• " - lyour pay ' ^ 

- what you're learning 
from the job 

- the help of the CAA 

4. What do you like most' about your job? 

5. Wliat do ypu like least? +1ave you 
,^ad any problems oh the job? 

9 

6. If we could improve the program, what 
would you recommend that we do?^ 



f 



J 



■ 7a 



ERIC 



